Catadioptric Omni-directional Stereo Vision and Its Applications in Moving Objects Detection

Catadioptric Omni-Directional Stereo Vision (ODSV) technology is the combination of omni-directional imaging and stereo vision, which has wide application potentials on robot vision and large-scale video surveillance [1-4]. Fig.1 gives the framework of ODSV technology, which includes four major parts: the design of omni-directional stereo vision imaging system, unwarping of omni-directional stereo images, rectification of omnidirectional stereo images, stereo matching and depth estimation of omni-directional stereo vision. Among these four parts, the imaging system can be used to capture omni-directional stereo image pair(s), which is the input of omni-directional stereo vision. An omni-directional stereo vision imaging system is typically composed of catadioptric mirrors, imaging sensors and fasteners. The purpose of unwarping the omni-directional stereo images is to convert the circularity shaped omni-directional images into perspective projection images, which are suitable for human watching. Generally, we call the circularity shaped images captured by catadioptric omni-directional imaging system as omni-directional images, and we call the unwarped images that are suitable for human watching as panoramic images. Rectification of omni-directional stereo images can be regarded as the pretreatment before stereo matching. In many cases, there are horizontal errors and vertical errors in the omnidirectional images and panoramic images, these errors result in large searching space and mismatching when performing stereo matching. The rectification of omni-directional stereo images uses epipolar geometry to transform the images, which makes the matching points lie on a horizontal scan line, and reduce the searching space from two-dimension to onedimension, so as to improve the stereo matching efficiency. Stereo matching and depth estimation of omni-directional stereo vision are key problems in catadioptric omni-directional stereo vision, whose main function is to find correspondences between pixels among a pair of or more reference images, i.e. to estimate relative disparity for each pixel in reference images. Given pixel correspondence and calibrated camera, it is easy to figure out the depth information via triangulation for the determinate relationship between disparity and depth. Taking its advantages of large FOV (Field of View) and depth information, catadioptric omni-directional stereo vision can be widely used in robot vision and video surveillance. For example, in robot football games, we can use this technology to make robots to "see" the O pe n A cc es s D at ab as e w w w .ite ch on lin e. co m